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PATENT APPLICATION 
0100.9900440 

METHOD AND APPARATUS FOR RENDERING AN 
IMAGE IN A VIDEO GRAPHICS ADAPTER 

Field of the Invention 

The present invention relates generally to the rendering of data in a video graphics 
system, and more particularly to a method and apparatus for rendering active video using 
more than one video adapter. 

Background of the Invention 

Figure 1 illustrates a prior art system for rendering video data in a computer 
system. Specifically, the prior art system of Figure 1 illustrates a video-input signal 
being received by a video decoder. Such a video-input signal could be from a television, 
VCR, DVD player, or compressed video data. The received video data is decoded, as 
necessary by the video decoder which and provides a video-output across the PCI bus to 
the video memory associated with the graphics adapter of Figure 1. The data stored 
within the video memory is then displayed on the graphics device which is also 
connected to the graphics adapter. 

The prior art configuration of Figure 1 can be inefficient because of the need to 
transport the video data across the PCI bus. In order for the decoded data to be stored 
within the graphics adapter's video memory, it is necessary for the video decoder to have 
its PCI bus control logic to store the rendered video information within the video 
memory. The hardware necessarily of the video decoder to interface to the PCI bus is 
costly in terms of space and design implementation. Another inefficiency of the system 
of Figure 1 is the use of PCI bandwidth by the video decoder when transmitting the data 
to the video memory. The video decoder is capable of performing the data transfer, and 



2 

does not require system processor intervention. However, the system processor can be 
stall if it needs to access the PCI bus during a transfer of video data by the video decoder. 
Therefore, the bandwidth used by the video decoder can prevent a system processor, or 
any other peripheral requiring the PCI bus, from functioning optimally when unable to 
access the PCI bus. For example, for a 320-by-240 pixel screen the number of bytes of 
data that needed to be transferred each second between the video decoder and the video 
memory would be at least 320 x 240 x 2 bytes x 60 frames per second. 

Figure 2 illustrates a second prior art solution to overcome these problems,. In the 
prior art system of FIG. 2, the video input is received by the video decoder in the same 
manner as discussed with Figure 1 . However, the decoded data is not transferred to the 
graphics adapter across the PCI bus. Instead, the decompressed data is transferred across 
a dedicated local bus. By transmitting the video data from the decoder to the graphics 
adapter across the dedicated bus, the bandwidth associated with its use of the PCI bus is 
eliminated, thereby freeing PCI bandwidth for other peripherals or CPU. In addition, the 
use of a dedicated local bus allows for the expensive PCI hardware associated with the 
video controller of Figure 1 to be avoided. 

In general, the control circuitry associated with the local bus of Figure 2 is 
inconsequential as compared to the overhead associated with the PCI bus of Figure 1 . 
The advantage of using the dedicated bus of the type in Figure 2 is that it requires the 
video decoder to be connected to a single graphics adapter. With computer systems, such 
as Windows 98, it is necessary for the video decoder to be associated with the primary 
adapter on the PCI bus in order to display active video, where the primary adapter is the 
video adaptor first identified in the hardware by the operating system. 

Therefore, a system that allows for a video-input signal to be displayed on a 
primary adapter, or any of a number of secondary video adapters, would be 
advantageous. 
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Brief Description of the Drawings 

Figure 1 illustrates in block diagram form a prior art implementation of a portion 
of a video system of a computer system; 

Figure 2 illustrates in block diagram form a prior art version of another portion of 
a prior art video system of a computer system; 

Figure 3 illustrates in block diagram form the video portion of a computer system 
in accordance with the present invention; 

Figure 4 illustrates in flow diagram form a method for implementing a graphics 
control method in accordance with the present invention; and, 

Figure 5 illustrates a block diagram of a data processing in which specific 
embodiments of the present invention can be implemented. 

It should be understood that the figures included herein illustrate specific 
embodiments of the present invention. Other embodiments of the present invention may 
exist. Specific elements illustrated with the embodiments herein are not intended to 
represent actual size, location relationships between the components. 

Detailed Description of the Drawings 

In accordance with the present invention, a video source is received by a first 
video adapter. The video source is captured in the video memory associated with the first 
VGA. The stored video source is associated with a window of an existing application. 
When the window of the existing application is shifted to coincide with the video 
memory of a second graphics adapter, the first graphics adapter performs a memory 
access function across a systems bus to the appropriate video memory location with the 
second graphics adapter to allow the rendering of that portion of the video now residing 
on a second monitor. Such remote rendering allows for active video signals to be 
displayed on any secondary or primary video graphics adapter. This is an advantage over 
the prior art, which precluded the displaying of active video graphics on secondary 
graphics adapters. 
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Figure 3 illustrates a portion of a computer system 300. The system 300 
comprises a video decoder 310, a first graphics adapter 320, and a second graphics 
adapter 330. The video decoder 310 is connected to the first graphics adapter through the 
dedicated local bus 340. The dedicated local bus 340 is connected to the video decoder 
3 1 0 at a port 3 1 1 , and the first graphics adapter at port 32 1 . The graphics adapter 320 is 
bi-directionally connected to system bus 350. The system bus 350 is illustrated to be a 
PCI bus, however any industry standard or proprietary bus capable of transmitting data at 
speeds appropriate to support video applications are anticipated by the present invention. 
The graphics adapter 320 has a video memory 322, which is accessed locally by the 
adapter 320, for storing video data to be displayed on the monitor 324 which is connected 
to the graphics adapter 320. The graphics adapter 330 is bi-directionally connected to the 
PCI bus 350, has its own local video memory 332, and is connected to a monitor 334 
where rendered data stored in video memory 322 is displayed. 

The portion 300 may include discrete add-on cards in a general purpose computer, 
components integrated on a mother board, such as such as Application Specific 
Integrated circuits (ASICs) or data processors. This bus 340 may be any number of 
connectors, including ribbon cable connecting two separate add-on boards, a bus 
integrated onto a mother board, or connector pins associated with a bus where the video 
decoder actually plugs into the graphics adapter 340. 

In operation, a video-in signal is received at the video decoder 310. The video-in 
signal can be representative of any number of video signals. For example, the video-in 
could be a compressed video signal such as an MPEG video signal, a DVD video signal, 
a video signal from a VCR, a television, or any other video source. The video decoder 
310 converts the video-in to a video source signal usable by the graphics adapter 320. 
Once the video-in conversion process is completed by the decoder 310, the video source 
is transmitted across the dedicated local bus 340 to the graphics adapter 320 In other 
embodiments, the dedicated video bus 340 could be connected to other peripheral boards 
as well. However, in accordance with this specific embodiment, video data will not be 
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transmitted across the system bus 350. The data transmitted across the bus 340 is 
captured into the video memory 322 by the graphics adapter 320. Once captured at the 
video memory 322, it is possible to retrieve the data from the video memory 322 and 
display it visually onto the monitor 324. 

During normal operation the system 300 will be used in conjunction with an 
operating system, such as Windows 98 by Microsoft. Applications run under such an 
operating system would display the active video data, received by the video decoder and 
subsequently captured in the video memory 322, within an application window on 
monitor 324. The application window is specified by the operating system, and the 
graphics adapter renders data from the video memory 322 such that it can be displayed 
within the application window on monitor 324. 

If the application window is moved to a different monitor, or a portion of the 
window is moved to a different monitor, such as monitor 334 associated with graphics 
adapter 330, the following sequence of events will occur. The operating system, in 
response to the user's inputs, would transmit operating system commands indicating the 
new window location. These operating system commands are interpreted by the graphics 
adapters 320 at memory 322 and 330. In response, the graphics adapter 330 will 
recognize that a portion of its video memory is to be displayed. However, all of the video 
data to be displayed is still being received and stored by the graphics adapter 320. 
Therefore, it is necessary for graphics adapter 320 to recognize the application window 
previously being displayed exclusively on monitor 324 is now at least partially being 
displayed on monitor 334. In response, the graphics adapter 320 will determine that 
portion of it video memory 322 that is now to be displayed by the adapter 330. This 
portion of the memory 322 will be sent to the adapter 330 using a transfer technique, such 
as a DMA transfer. DMA hardware capable of transmitted the video to the appropriate 
video memory location in video memory 332 can be located on VGA 320. 

Once the video data associated with the screen 334 is stored within the video 
memory 332, it is possible for the graphics adapter 330 to render this data as an image on 
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the monitor 334. Note that in one embodiment, as video input continues to be received at 
the decoder, and transmitted across the dedicated local bus 340 to the graphics adapter 
320, it would be necessary for a DMA controller within the graphics adapter 320 to 
continue to transfer the data from the video memory 322 across the PCI bus to the video 
memory 332 of graphics adapter 330. In this manner, it is possible for video data to be 
displayed on multiple display devices. 

In one embodiment of the present invention, the video source data received on a 
dedicated local bus 340 will continue to be stored in the video memory 322, and the 
portion of video memory 320 needed at the remote graphics adapter 330 would be 
transmitted across the PCI bus as received and/or needed. In another embodiment, it 
would be possible to incorporate appropriate hardware to the graphics adapter 320 
capable of monitoring data being received across the remote local bus 340, and 
transferring data that is rendered only on the adapter 330 directly to the adapter 330 
without first saving it to memory 322. In a specific embodiment where video is to be 
mirrored on the monitors 324 and 334, it would be necessary for the video memory 322, 
and the video memory 332 to contain the same data information. Note, however, the 
video information stored within each of the two memories 322 and 332 would not 
necessarily be stored at the same memory location within each of the respective 
memories 322, and 332. 

In accordance with the present invention, either graphics adapter 320, or the 
graphics adapter 330 may be the primary graphics adapter. The term "primary graphics 
adapter" refers specifically to the plug-and-play terminology of Windows 98 operating 
system, where the first video graphics adapter recognized during the start-up routine is 
considered the primary video adapter by the operating system. Under Windows 98, only 
a primary adapter can be operationally connected to a video decoder as indicated in 
Figure 3. An advantage of the present invention is that one or more secondary video 
adapters may actually be connected to a video decoder 310 for receiving video data. 
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Yet another advantage of the present invention is that multiple video sources can 
be associated with the system 300. For example, a second video source 315 can be 
connected through its own dedicated bus to the graphics adapter 330, The video source 
315 can be a second video decoder, television, videocassette recorder, or any other video 
source. The second video source 315 provides a video-out signal (a video source) to the 
adapter 330, which would be a video source relative to the adapter 330. In this situation, 
it would be possible for the multiple video-in sources to be displayed upon any of the 
monitoring devices of the present system. It should also be understood by one skilled in 
the art that the present invention is not limited to two graphics adapters or two video 
decoders. For example, a video wall concept could be used. A video wall refers to a 
technique whereby multiple display devices are stacked on top and beside each other in 
an array fashion to display a video image on a large scale. For example, four monitors 
can be arranged in a 2-by-2 grid, where each monitor would be displaying approximately 
one-quarter of the video-in signal. Using this technique, the video-in signal would be 
received by the video decoder 310 transmitted across dedicated bus 340 to the graphics 
adapter 320 and stored in its memory 322. Next, the DMA associated with the graphics 
adapter 321 will transmit one-quarter of the data stored in video memory 322 to a first 
video memory, a second quarter to a second video memory, and the third quarter to a 
third video memory. Thereby, each of the video memories would have approximately a 
quarter of the data, and all of the data it needs to display its quadrant on the wall monitor. 

Figure 4 illustrates a data processing system 400, such as may be used to 
implement the present invention, and would be used to implement the various 
methodologies, or incorporate the various hardware disclosed herein. 

The system of FIG. 4 includes a central processing unit (CPU) 410, which may be 
a conventional or proprietary data processor, and a number of other units interconnected 
via system bus 402. 

The other units include random access memory (RAM) 412, read-only memory 
(ROM) 414, and input/output (I/O) adapter 422 for connecting peripheral devices, a user 
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interface adapter 420 for connecting user interface devices, a communication adapter 424 
for connecting the system 400 to a data processing network, and a video/graphic 
controller for displaying video and graphic information. 

The I/O adapter is further connected to disk drives 447, printers 445, removable 
storage devices 446, and tape units (not shown) to bus 402. Other storage devices may 
also be interface to the bus 412 through the I/O adapter 422. 

The user interface adapter 420 is connected to a keyboard device 440 and a mouse 
441. Other user interface devices such as a touch screen device (not shown) may also be 
coupled to the system bus 402 through the user interface adapter 420. 

A communication adapter 424 connected to bridge 450 and/or modem 451 . 
Furthermore, a video/graphic controller 426S connects the system bus 402 to a display 
device 460. 

In operation, the bus 402 could correspond to the PCI bus 350 of FIG. 3, and the 
video graphics adapters would be connected to bus 402 in the same manner as controller 
426. In addition, methods associated with the present invention, if any, may be 
implemented and stored on computer readable media such as one or more of the storage 
device 445, 446, and $47 for subsequent processing by CPU 410. Since the apparatus 
implementing the present invention is, for the most part, composed of electronic 
components and circuits known to those skilled in the art, circuit details will not be 
explained in any greater extent than that considered necessary as illustrated above, for the 
understanding and appreciation of the underlying concepts of the present invention and in 
order not to obfuscate or distract from the teachings of the present invention. 

Figure 5 illustrates a method in accordance with the present invention. At step 
510, an operating system start-up procedure occurs. It is during such a start-up 
procedure, that the operating system determines what hardware is present in the system, 
and in response, defines the system including a system memory map. For example, with 
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reference to Figure 3, the operating system start-up procedure needs to recognize that two 
graphics adapters 320 and 330 exist in the system, and that the graphics adapter 320 has a 
video decoder 310 associated with it. In addition, the start-up procedure would also 
recognizes the amount of video memory associated with the first graphics adapter and the 
amount of video memory 322 associated with the second video graphics adapter. Based 
upon this information, the video memories 322 and 332 would be memory mapped into 
the PCI bus 350. 

One method of recognizing the video decoder 3 10 on remote bus 340 is put forth 
in Patent Application having attorney docket number 9900300, filed on February, XX, 
1999, and having Application Number 09/XXX,XXX (To be assigned), and is hereby 
incorporated by reference. 

Next, in step 51 1, an application start-up occurs. In a Specific embodiment, the 
application is an active video application whereby an active video signal is received and 
displayed within a window opened by, or for, the application. For example, if a user 
chose to watch a television program on a computer screen, an application capable of 
displaying such a television program would be executed. 

During start-up of the video application, a window would be defined and 
transmitted across the system by the operating system. As part of the system information, 
the first video graphics controller will interpret the operating system information and, as a 
result, begin storing the actual video source information within its memory for display in 
the window itself. With an operation system such as Windows 98, this would be 
accomplished by the primary video graphics adapter. Therefore, the primary video 
graphics adapter will capture the data, and render it onto the screen in the window 
defined by the system. 

Next, at step 512, a window move command is interpreted by a second video 
graphics adapter. This occurs when a user input defines that a window is supposed to be 
displayed at least partially on a second display device. For example, if a user drags a 



10 



window displaying active video graphics from a first monitor across to a second monitor, 
a command would be transmitted across the operating system and intercepted by both the 
first video graphics adapter and the second video graphics adapter. In other 
embodiments, it would be possible to display a portion of the window on each of the two 
display devices, or it would be possible even to have multiple windows displaying the 
same video data on multiple systems. 

Once the first video system recognizes that a portion of the window has moved to 
a monitor controlled by a different video controller, it will send the captured video to the 
second VGA. Generally, this would be accomplished across the system bus, such as the 
PCI bus. This is done generally by a direct memory access (DMA) type device that is 
controlled by the adapter that monitoring systems calls, and is aware of the new location 
in the other adapter where to map the captured data. One of ordinary skill in the art will 
recognize that in other implementations, instead of having a DMA sending the captured 
data to the second video graphics adapters memory, it would be possible to intercept the 
data before it is stored in the first VGA's memory, thereby keeping just one copy at the 
location needed (the second VGA). 

Next, at step 514, the video data associated with each of the VGA's, now stored in 
their respective local memories, is rendered in a normal manner such that it is displayed 
on a screen. 

The present invention has been put forth in terms of specific embodiments not 
intended to be limiting to the invention itself. Other specific implementations of the 
present invention are possible. For example, in FIG. 3, bus other than a PCI bus may be 
used. The present invention is advantageous over the prior art in that a video-in signal 
can be display by any graphics adapter in the system. 
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I Claim: 



1. A method of displaying active video on a computer system, the method comprising the steps 
of: 

receiving at a first video graphics adapter (VGA) a first frame of active video from a video 
source; 

rendering at least a first portion of the first frame of video at the first VGA in response to a 
first control signal; 

rendering at least a second portion of the first frame of video at a second VGA in response to 
a second control signal. 

2. The method of claim 1, wherein the first portion and the second portion are the same portion. 

3. The method of claim 1, wherein the step of rendering at least a first portion of the first frame 

of video at the first VGA includes storing the at least a first portion of the active decoded 
video in a video memory associated with the first VGA. 

4. The method of claim 3, wherein the step of rendering at least a second portion of the first 

frame of video at the second VGA includes the substep of: 

storing the at least second portion of the active decoded video in a first video 
memory associated with the first VGA. 

5. The method of claim 4 further including the substep of: 

reading the second portion of the active decoded video from the first video memory and 

storing the at least second portion of the active decoded video in a first video memory 
associated with the first VGA. 



6. The method of claim 5, wherein the first video memory and second video memory are 

accessed by a direct memory access (DMA) controller associated with the first VGA. 
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7. The method of claim 5, wherein the first video memory and second video memory are 

accessed by a direct memory access (DMA) controller on the second VGA. 

8. The method of claim 1 , wherein the first VGA is a primary VGA, and the second VGA is a 

secondary VGA. 

9. The method of claim 1, wherein the first VGA is a secondary VGA, and the second VGA is a 

primary VGA. 

10. The method of claim 1, wherein the first VGA and the second VGA are part of a video wall 

such that the first frame of active video is displayed across multiple displays 
simultaneously. 

11. The method of claim 1 further comprising the steps of: 

receiving at the second VGA a second frame of active video from a second video source; 
rendering at least a portion of the second frame of video at the first VGA. 

12. The method of claim 1, wherein the first control signal is a signal specifying a window 

location for displaying the active video. 

13. The method of claim 12 further comprising the step of storing the window location in a 

preference file. 
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14. A processing system for executing instructions, the processor system comprising instructions 

for: 

monitoring the location of an active video window; 
storing active video data at first video memory; 

sending the active video data from the first video memory to a second video memory 
when the location of the active video window is associated with the second 
video memory. 

15. A method of displaying active video on a computer system, the method comprising the steps 
of: 

receiving at a first video graphics adapter (VGA) a first frame of active video from a video 
source 

displaying at least a first portion of the first frame of video at a second VGA in response to a 
second control signal. 

16. The method of claim 15, wherein the method further comprises the video source being a 

video decoder. 

17. The method of claim 16, wherein the video decoder is for decoding a compressed video 

signal. 

18. The method of claim 16, wherein the method further comprises the video source sending the 

first frame of data over a bus local to the first VGA. 

1 1 9. The method of claim 15, wherein the method further comprises storing the first frame of 

2 active video in a video memory associated with the first VGA. 
3 

1 20. The method of claim 1 5, wherein the method further comprises the video source being a 

2 television signal. 
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Abstract of the Disclosure 

In accordance with the invention, a video source is received by a first video 
adapter. The video source is captured in the video memory associated with the first 
5 VGA. The stored video source is associated with a window of an existing application. 
When the window location of the existing application is shifted to coincide with the video 
memory of a second graphics adapter, a data transfer occurs to the appropriate video 
memory location of the second graphics adapter to allow the rendering of that portion of 
the video now residing on a second monitor. 
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Facsimile: 312-939-9828 

I hereby declare that all statements made herein of my own knowledge are true and that all statements made on 
information and belief are believed to be true; and further that these statements were made with the knowledge that 
willful false statements and the like so made are punishable by fine or imprisonment, or both, under 18 U.S.C. 1001 and 
that such willful false statements may jeopardize the validity of the application or any patent issued thereon. 



Name of Sole or First Inventor: 



Given Name (first and middle ["if any]) 



□ A petition has been filed for this unsigned inventor 
Family Name or Surname 



%a 



Klebanov 



Inventor's 

Signature 

Residence [ City:Vaughan X/ 

Post Office Address j 202 Hnewood Drive 



Date 



State: Ontario j Country: Canada 1 Citizenship: Israel 



City: Vaughan 



state: Ontario 



ZIP: L4J 5R6 



1 Country: Canada 



Name of Additional Joint Inventor: 


Zl A petition has been filed for this unsigned inventor 


Given Name (first and middle [if any]) 


Family Name or Surname 








Inventor's 
Signature 


Date 


Residence 


City: state: 


Country: Citizenship: 


Post Office Address | 


City: State: 


ZIP: Country: 



Name of Additional Joint Inventor: 



Given Name (first and middle ["if any]) 



□ A petition has been filed for this unsigned inventor 
J Family Name or Surname 



Inventor's 
Signature 



Residence 



City: 



State: 



Date 



Country: 



Citizenship: 



Post Office Address 
City: 



State: 



ZIP: 



| Country: 



□ Additional inventors are being named on the supplemental Additional Inventor(s) sheet(s) PTO/SB/02A attached hereto. 



